Distributed Robust Learning

We propose a generic distributed learning framework for robust statistical learning on big contaminated data. The Distributed Robust Learning (DRL) framework can reduce the computational cost of traditional robust learning methods by several orders. We provide a sharp analysis on the robustness of DRL, showing that DRL not only preserves the robustness of base robust learning methods, but also tolerates breakdowns of a constant fraction of computing nodes. Moreover, DRL can enhance the breakdown point of existing robust learning methods to be even larger than 50%, under favorable conditions. This enhanced robustness is in sharp contrast with the naive divide and fusion method where the breakdown point may be reduced by several orders. We specialize the DRL framework for two concrete cases: distributed robust PCA and distributed robust regression. We demonstrate the efficiency and the robustness advantages of DRL through comprehensive simulations.